Studies with Fabricated Switchboard Data: Exploring Sources of Model-data Mismatch

We present a study of data simulated using acoustic models trained on Switchboard data, and then recognized using various Switchboard-trained acoustic models. The Switchboard-trained models yield word error rates of about 47 percent, on real Switchboard conversations. When data is simulated using the acoustic models, but in a way that insures that the pronunciations in our recognition dictionary are “perfect”, the WER drops by nearly a factor of five. If instead we use hand-labeled phonetic transcriptions to fabricate data that more realistically represents the way words are pronounced – rendering our recognition pronunciations imperfect – we obtain WERs in the low 40’s, rates that are fairly similar to those seen in actual speech data. Taken as a whole, these and other experiments we describe in the paper suggest that there is a substantial mismatch between real speech data and our speech models. The use of simulation in speech recognition research appears to be a promising tool in our efforts to understand and reduce the size of this mismatch.